Extreme Fluctuations in Small- Worlds with Relaxational Dynamics 
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We study the distribution and scaling of the extreme height fluctuations for Edwards- Wilkinson- 
type relaxation on small-world substrates. When random links are added to a one-dimensional 
lattice, the average size of the fluctuations becomes finite (synchronized state) and the extreme 
height diverges only logarithmically in the large system-size limit. This latter property ensures 
synchronization in a practical sense in small-world coupled multi-component autonomous systems. 
The statistics of the extreme heights is governed by the Fisher- Tippett-Gumbel distribution. 

PACS numbers: 89.75.Hc, 05.40.-a, 89.20.Ff, 
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Synchronization is a fundamental problem in natural 
and artificial coupled multi-component systems 1|. Since 
the introduction of small-world (SW) networks [2j it has 
been well established that such networks can facilitate 
autonomous synchronization [lj . Examples include noisy 
coupled phase oscillators |3| and scalable parallel simula- 
tors for asynchronous dynamics 0. In essence, the SW 
coupling introduces an effective relaxation to the mean 
of the respective local field variables (or local "load"), 
and induces (strict or anomalous) mean-field-like behav- 
ior 00. In addition to the average load in the network, 
knowing the typical size and the distribution of the ex- 
treme fluctuations 0> II El is of great importance from 
a system-design viewpoint, since failures and delays are 
triggered by extreme events occurring on an individual 
node. 

In this Letter, we focus on the steady-state proper- 
ties of the extreme fluctuations in SW-coupled interact- 
ing systems with relaxational dynamics. In contrast, 
consider, for example, kinetically growing possibly non- 
equilibrium surfaces with only short-range interactions 
(e.g., nearest neighbors on a lattice). Here a suitably 
chosen local field variable is the local height fluctuation 
measured from the mean E3- It was shown E3 that 
in the steady state, where the surface is rough, the ex- 
treme height fluctuations diverge in the same power-law 
fashion with the system size as the average height fluc- 
tuations (the width). Similar observation was made fl^] 
in the context of the scalability of parallel discrete-event 
simulations (PDES) [H Q E3, where the progress of 
the simulation is governed by the Kardar-Parisi-Zhang 
(KPZ) equation |l6|: here the "relative height" or local 
field variable is the deviation of the progress of the in- 
dividual processor from the average rate of progress of 
the simulation |l5l |. The systems in the above exam- 
ples are "critical" in that the lateral correlation length 
of the corresponding rough surfaces scales with the sys- 
tem size [lo|. For systems at criticality with unbounded 
local variables, the extreme values of the local fluctu- 
ations emerge through the dominating collective long- 
wavelength modes, and the extremal and the average 



fluctuations follow the same power-law divergence with 
the system size ^1 0] . Relationship between extremal 
statistics and universal fluctuations in correlated systems 
have been studied intensively [H El El El El El El • 
Here we discuss to what extent SW couplings (extending 
the original dynamics through the random links) lead to 
the suppression of the extreme fluctuations. We illus- 
trate our findings on an actual synchronization problem 
for scalable PDES schemes 0. 

First, we briefly summarize the basic properties of the 
extremal values of N independent stochastic variables 
H H H El E3- Here we consider the case when the 
individual complementer cumulative distribution P>(x) 
(the probability that the individual stochastic variable is 
greater than x) decays faster than any power law, i.e., ex- 
hibits an exponential-like tail in the asymptotic large- x 
limit. (Note that in this case the corresponding prob- 
ability density function displays the same exponential- 
like asymptotic tail behavior.) We will assume P>(x) ~ 
e -cx r or i ar g e x va i ueSj where c is a constant. Then the 
cumulative distribution P™ ax (cc) for the largest of the N 
events (the probability that the maximum value is less 
than a;) can be approximated as [22L l2.'"il| 
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where one typically assumes that the dominant con- 
tribution to the statistics of the extremes comes from 
the tail of the individual distribution P> (x) . With the 
exponential-like tail in the individual distribution, this 
yields 



pmax 



(X) 



(2) 



The extreme- value limit theorem states that there exists 
a sequence of scaled variables x — (x — ajv)/&iv, such 
that in the limit of N^oo, the extreme- value probability 
distribution for x asymptotically approaches the Fisher- 
Tippett-Gumbel (FTG) distribution 0,01 : 



P^ ax (x) 



(3) 



with mean (i)=7 (7=0.577... being the Euler con- 
stant) and variance <r| = (x 2 ) — (i) 2 =7r 2 /6. From 



Eq. J3J, one can deduce that to leading order 
the scaling coefficients must be aAr=[ln(7V)/c] 1 /' 5 and 
& A r=(<5c)- 1 [ln(iV)/c]( 1 /^- 1 [HEl. The average value 
of the largest of the N original variables then scales as 

<x max ) = a N + b Nl ~ [ln(iV)/c] V* (4) 

(up to 0( in^jv) ) correction) in the asymptotic large- TV 
limit. 

We now consider the Edwards- Wikinson (EW) model 
|2fil |. a prototypical synchronization problem with rc- 
laxational dynamics 0, 0, 0], first on a regular one- 
dimensional lattice: d^hi = + Here 
hi is the local height (or load) at site i, —^=6^1 + 
<5,j_i — 2Sij is the discrete Laplacian on the lattice, 
and rji(t) is a short-tailed noise (e.g., Gaussian), delta- 
correlated in space and time. The width, borrowing 
the framework from non-equilibrium surface-growth phe- 
nomena, provides a sensitive measure for the average de- 
gree of s ync hronization in coupled multi-component sys- 
tems mUm. The EW model on a one-dimensional lat- 
tice with N sites has a "rough" surface profile in the 
steady state (de-synchronized state), where the average 
width (ui 2 )at=(-^ J2i=i(hi ~ h) 2 ) diverges in the ther- 
modynamic limit as (w 2 )^N. (Here, h=(l/N)^2^_-.hi 
is the mean height.) The diverging width is related to 
an underlying diverging lengthscale, the lateral correla- 
tion length, which reaches the system size N for a fi- 
nite system. Further, the maximum height fluctuations 
(measured from the mean height h) diverges in the same 
fashion as the width itself, i.e., ((h m& x — h) 2 )~N [lllll2] |. 

Here we ask how the scaling behavior of the extremal 
height fluctuations changes if we extend the same dy- 
namics to a SW network Q ■ Then the equation of mo- 
tion becomes d t hi = — Yljfiij + + Vi(t)> where 
—Vij — Jij — Sij J2i Ju i s the Laplacian on the random 
part of the network. The symmetric matrix repre- 
sents the quenched random links on top of the regular 
lattice, i.e., it is 1 if site i and j are connected and oth- 
erwise. In a frequently studied version of the SW network 
HE Ell HH random links of unit strength are added to 
all possible pair of sites with probability p/N ( "soft" net- 
work 0). Here p becomes the average number of random 
links per site. In a somewhat different construction of the 
network ("hard" network 0,0), each site has exactly one 
random link (i.e., pairs of sites are selected at random, 
and once they are linked, they cannot be selected again) 
and the strength of the interaction through the random 
link is p. Note that both SW constructions have a fi- 
nite average degree for each node, and are embedded in 
a finite dimension. The common feature in both SW ver- 
sions is an effective nonzero mass (in a field theory 
sense), generated by the quenched-random structure [6j. 
In turn, both the correlation length £~[£(p)] -1 / 2 and the 
width (ui 2 )~(l/2)[S(p)] -1 / 2 approach a finite value (syn- 
chronized state) and become self-averaging in the iV— >oo 



limit. For example, for the EW model on the soft and 
hard SW versions in one dimension, for small p values, 
£(p)~p 2 and S(p)~p, respectively gg. Thus, the corre- 
lation length becomes finite for an arbitrarily small but 
nonzero density of random links (soft network), or for 
an arbitrarily small but nonzero strength of the random 
links (one such link per site, hard network). 

This is the fundamental effect of extending the original 
dynamics to a SW network: it decouples the fluctuations 
of the originally correlated system. Then, the extreme- 
value limit theorems can be applied u sing th e number of 
independent blocks N/£ in the system |22j,|23J. Further, if 
the tail of the noise distribution decays in an exponential- 
like fashion, the individual relative height distribution 
will also do so |29|, and depends on the combination 
Ai/w, where Ai=hi—h is the relative height measured 
from the mean at site i, and w=yj (w 2 ). Considering, 
e.g., the fluctuations above the mean for the individual 
sites, we will then have P>(Aj) ~ exp[— c(Ai/w) s ], where 
P>(Ai) denotes the "disorder-averaged" (averaged over 
network realizations) single-site relative height distribu- 
tion, which becomes independent of the site i for SW 
networks. From the above it follows that the cumulative 
distribution for the extreme-height fluctuations relative 
to the mean A rT1RY = fe rn a Y — h, if scaled appropriately, will 
be given by Eq. (0) 30] in the asymptotic large- iV limit 
(such that iV/£»l). Further, from Eq. gj), the average 
maximum relative height will scale as 

(A m ^, W (^y\^ M N)f S , (5) 

where we dropped all TV-independent terms. Note, 
that both w and £ approach their finite asymptotic N- 
independent values for SW-coupled systems, and the only 
iV-dependent factor is ln(iV) for large N values. Also, 
the same logarithmic scaling with N holds for the largest 
relative deviations below the mean (h—h m i n ) and for 
the maximum spread (h max — /tmin)- This is the central 
point of our Letter: in small-world synchronized systems 
with unbounded local variables driven by exponential-like 
noise distribution (such as Gaussian), the extremal fluc- 
tuations increase only logarithmically with the number 
of nodes. This weak divergence, which one can regard as 
marginal, ensures synchronization for practical purposes 
in coupled multi-component systems. 

Next we illustrate our arguments on a SW- 
synchronized system, the scalable parallel discrete-event 
simulator for asynchronous dynamics . Consider an ar- 
bitrary one-dimensional system with nearest-neighbor in- 
teractions, in which the discrete events (update attempts 
in the local configuration) exhibit Poisson asynchrony. In 
the one site per processing element (PE) scenario, each 
PE has its own local simulated time, constituting the vir- 
tual time horizon {hi{t)}f =1 (essentially the progress of 
the individual nodes). Here t is the discrete number of 
parallel steps executed by all PEs, which is proportional 
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to the wall-clock time and N is the number of PEs. Ac- 
cording to the basic conservative synchronization scheme 
[lij . at each parallel step t, only those PEs for which the 
local simulated time is not greater then the local sim- 
ulated times of their virtual neighbors, can increment 
their local time by an exponentially distributed random 
amount. (Without loss of generality we assume that the 
mean of the local time increment is one in simulated time 
units [stu].) Thus, denoting the virtual neighborhood of 
PE i by Si, if hi(t) < minj e ,sr 4 {/ij-(i)}, PE i can update 
the configuration of the underlying site it carries and de- 
termine the time of the next event. Otherwise, it idles. 
Despite its simplicity, this rule preserves unaltered the 
asynchronous causal dynamics of the underlying system 
Ilfij . In the original algorithm 0] 1 the virtual com- 
munication topology between PEs mimics the interac- 
tion topology of the underlying system. For example, for 
a one-dimensional system with nearest-neighbor interac- 
tions, the virtual neighborhood of PE i, Si, consists of 
the left and right neighbor, PE i— 1 and PE It was 
shown ^| that then the virtual time horizon exhibits 
KPZ-like kinetic roughening and the steady-state behav- 
ior in one dimension is governed by the EW Hamiltonian. 
Thus, the average width of the virtual time horizon (the 
spread in the progress of the individual PEs) diverges 
as N^oo [ljj . hindering efficient data collection in the 
measurement phase of the simulation • To achieve a 
near-uniform progress of the PEs without employing fre- 
quent global synchronizations, it was shown |J] that in- 
cluding randomly chosen PEs (in addition to the nearest 
neighbors) in the virtual neighborhood results in a finite 
average width. Here we demonstrate that SW synchro- 
nization in the above PDES scheme results in logarith- 
mically increasing extreme fluctuations in the simulated 
time horizon, governed by the FTG distribution. 

In one implementation of the above scheme, each PE 
has exactly one random neighbor (in addition to the near- 
est neighbors) and the local simulated time of the random 
neighbor is checked only with probability p at every sim- 
ulation step. The corresponding communication network 
is the hard version of the SW network, as described ear- 
lier, and the "strength" of the random links is controlled 
by the relative frequency p of the basic synchronizational 
steps through those links. In an alternative implementa- 
tion, the communication topology is the soft version the 
SW network (with p average number of random links per 
PE) , and the random neighbors are checked at every sim- 
ulation step together with the nearest neighbors. Note 
that in both implementations (where the virtual neigh- 
borhood Si now includes possible random neighbor(s) for 
each site i), the extra checking of the simulated time of 
the random neighbor is not needed for the faithfulness 
of the simulation. It is merely introduced to control the 
width of the time horizon Q. 

To study the extreme fluctuations of the SW- 
synchronized virtual time-horizon, we "simulated the 
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FIG. 1: Disorder-averaged probability densities for the scaled 
(zero mean and unit variance) extreme-height fluctuations for 
the SW-synchronized PDES time horizons, (a) For the hard 
SW network with p=0.10 and (b) for the soft SW network 
with p=0.50, on log-normal scales for three system sizes. The 
solid curve corresponds to the similarly scaled FTG density 
[3(j for comparison. 



simulations", i.e., the evolution of the local simulated 
times based on the above exact algorithmic rules. By 
constructing histograms for Aj, we observed that the tail 
of the disorder-averaged individual relative-height distri- 
bution decays exponentially for both SW constructions. 
Then, we constructed histograms for the scaled extreme- 
height fluctuations. The results, together with the sim- 
ilarly scaled FTG density [3(| , are shown in Fig. We 
also observed that the distribution of the extreme values 
becomes self-averaging, i.e., independent of the network 
realization. Finally, Fig. [21 shows that for sufficiently 
large N (such that w essentially becomes system-size in- 
dependent) the average (or typical) size of the extreme- 
height fluctuations diverge logarithmically, according to 
Eq. J5J) with 5=1. We also found that the largest relative 
deviations below the mean (h— h m i n ), and the maximum 
spread (h max — h mm ) follow the same scaling with the sys- 
tem size N. Note, that for our specific system (PDES 
time horizon), the "microscopic" dynamics is inherently 
non-linear, but the effects of the non-linearities only give 
rise to a renormalized mass S(p) (leaving X(p)>0 for all 
p>0) 0, H3- Thus, the dynamics is effectively governed 
by relaxation in a small world, yielding a finite correlation 
length and, consequently, the slow logarithmic increase of 
the extreme fluctuations with the system size [Eq. iJSJ]. 
Also, for the PDES time horizon, the local height distri- 
bution is asymmetric with respect to the mean, but the 
average size of the height fluctuations is, of course, finite 
for both above and below the mean. This specific charac- 
teristic simply yields different prefactors for the extreme 
fluctuations [Eq. JSJ] above and below the mean, leaving 
the logarithmic scaling with N unchanged. 

In summary, we considered the extreme-height fluc- 
tuations in a prototypical model with local relaxation, 
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FIG. 2: Average maximum relative height and average width 
for the SW-synchronized time horizon as a function of the 
number of nodes (PEs). Filled symbols: hard SW network 
with p=0.10. Open symbols: soft SW network with p=0.50. 
Note the normal-log scales. 



unbounded local variables, and short-tailed noise. We ar- 
gued, that when the interaction topology is extended to 
include random links in a SW fashion, the statistics of the 
extremes is governed by the FTG distribution. This find- 
ing directly addresses synchronizability in generic SW- 
coupled systems where relaxation through the links is 
the relevant node-to-node process and effectively governs 
the dynamics. We illustrated our results on an actual 
synchronizational problem in the context of scalable par- 
allel simulations. Analogous questions for heavy-tailed 
noise distribution and different types of networks have 
relevance to various transport and transmission phenom- 
ena in natural and artificial networks [3lj . For example, 
heavy-tailed noise typically generates similarly tailed lo- 
cal field variables through the collective dynamics. Then, 
the largest fluctuations can still diverge as a power law 
with the system size (governed by the Frechet distribu- 
tion 0,0), motivating further research for the properties 
of extreme fluctuations in complex networks 32]. 
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